This specification relates to microscope-based screening of biological material. More particularly, the specification concerns the acquisition of images at each of a plurality of depths along an optical axis in a three-dimensional specimen. Alternatively, the specification concerns the acquisition of images at each of a plurality of planes spaced along an optical axis in a three-dimensional specimen. Preferably, one object plane is acquired for each of a plurality of colors which are focused by a lens at different focal points in the space occupied by a three-dimensional specimen.
Revealing fundamental molecular mechanisms of diseases requires microscope-based screening of biological material. When the biological material is supported on a microscope slide, a multiwall plate, or other equivalent device, and the optical information obtained from images acquired by the microscope is utilized to make measurements of cells, the screening process is referred to as “image cytometry”; if the slide or plate is scanned to acquire a succession of images and cell measurements at a corresponding succession of locations, the process is referred to as “scanning cytometry”. Automated scanning cytometry is scanning cytometry enhanced by automation of system functions such as scanning, focus, image acquisition and storage, and so on. In automated scanning cytometry, high-resolution scans require medium-to-high numerical aperture (NA) objectives which have depths of field similar to the thicknesses of cell monolayers. To be able to focus such a layer for quantitative and qualitative analysis, autofocusing helps to achieve good image quality.
A cytometer is an instrument designed to count cells and measure their properties in order to: obtain quantitative and qualitative information for biomedical studies. Scanning cytometry, in contrast to flow cytometry, utilizes stage scanning of cells and tissues attached to substrates in synchronism with fully automated image processing to make quantitative measurements of large cell populations. Moreover, the cell classification is more precise than in flow cytometry since the cells that are attached in-situ maintain their normal shapes and are not suspended in a fluid stream. However, the increasing precision is limited by lower speed. A flow cytometer can have a scanning rate of 20-30 kHz (cells/second), whereas an image cytometer typically works at 100 Hz. To close the gap in scanning speed, especially for high-resolution scans that require high numerical objectives, on-the-fly auto focusing has been developed to ensure high image quality during high speed scanning.
In this regard, in contrast to classical flow cytometry, image cytometry has the advantage of a two-dimensional (2-D) representation of image information, containing a quantity of cells and their relocation (one flow cytometer, the ImageSream by Amnis, http://www.amnis.com/, also images cells in a flow stream). The 2-D extension has had the disadvantage of lower speed in comparison to the one-dimensional (1-D) flow cytometry. (Bravo-Zanoguera, M. E. & Price, J. H. Simultaneous Multiplanar Image Acquisition in Light Microscopy. SPIE Proc. Optical Diagnostics of Biological Fluids and Advanced Techniques Analytical Cytology 3260 (1998)). However, to detect ultra rare cells, such as cancer cells, which is an important part in biomedical applications, scanning image cytometry is currently the best method because it offers the opportunity to analyze the specimen in greater detail using high resolution images. In order to identify those ultra rare cells within large cell populations, it is desirable that this process be automated. (Bajaj, S., Welsh, J. B., Leif, R. C. & Price, J. H. Ultra-rare-event detection performance of a custom scanning cytometer on a model preparation of fetal nRBCs. Cytometry 39, 285-294 (2000); Morelock, M. M., et al. Statistics of assay validation in high throughput cell imaging of nuclear factor kappaB nuclear translocation. Assay Drug Dev Technol 3, 483-499 (2005); and, Prigozhina, N. L., et al. Plasma membrane assays and three-compartment image cytometry for high content screening. Assay Drug Dev Technol 5, 29-48 (2007)).
To accomplish automated scanning cytometry, the stage movement, specimen illumination, and image acquisition have to be automated by a host computer. To be able to classify individual cells small features, such as cell borders, have to be distinguished. See M. Bravo-Zanoguera: “Automatic on-the-fly focusing for continuous image acquisition in high-resolution microscopy”, Proc. Opt. Diag. Living Cells, pp. 243-252, 1999. For this reason, objectives with a high numerical aperture (e.g. 0.7) and magnification (e.g. 40×) are needed. Due to the increased lateral resolution and corresponding reduction in depth of field it is necessary to refocus the specimen for each field of view. Images taken out of focus are not able to offer sufficient information since a stable auto focus algorithm for medium and high re the image quality is reduced. (Bravo-Zanoguera, M. E. Dissertation, University of California (2001)). Therefore, a fully automated scanning cytometry system needs solution scans. (Oliva, M. A., Bravo-Zanoguera, M. & Price, J. H. Filtering out contrast reversals for microscopy autofocus. Applied Optics 38, 638-646 (1999)).
Automated cytometers are currently used in clinical and industrial applications. Clinical applications include screening systems (e.g. prescreening of cervical cancer), rare event detection systems (e.g. genetic screening) and hematology systems. Industrial applications include pharmaceutical (e.g. drug screening); food and cosmetics (e.g. bacteria count analysis) as well as routine work in biomedical research. (Bravo-Zanoguera; M. E. & Price, J. H. Simultaneous Multiplanar Image Acquisition in Light Microscopy. SPIE Proc. Optical Diagnostics of Biological Fluids and Advanced Techniques Analytical Cytology 3260 (1998)).
Incremental scanning, which requires the stage to be motionless while acquiring an image, is used in conventional automated cytometers to keep track of the focus while the entire specimen is scanned. In incremental scanning, a motorized mechanism also moves the specimen (or the objective) perpendicular to the optical axis (z-direction) to collect an axial series of images, also termed a “z-stack” of optical sections. Phase contrast, fluorescence, differential interference contrast (DIC) and other contrast enhancement mechanisms can be used to test for the sharpest image and adjust focus. After autofocusing, a fluorescence (typically, but also sometimes a bright field, phase contrast, DIC or other) image is acquired for the purpose of measuring cell properties. The first field of each specimen is focused manually (or with a longer automatic search range as is routine in drug screening with each new well) and the image is acquired. All subsequent fields are adjusted automatically during the use of an objective positioner (PIFOC, Physic lnstrumente http://www.physikinstrumente.com/en/products/prdetail.php?sortnr=200375), or by motorized focus actuators sold with the microscope (e.g., Nikon model Eclipse TI-e, http://www.nikoninstruments.com/ti/), or using a plate positioner (http://www.physikinstrumente.com/en/products/prdetail.php?sortnr=201546, or Mad City Labs, http://www.madcitylabs.com/nanoz500.html). This positioner is moved through a specific number of steps (Δz) in the z-direction. The autofocus is performed by collecting and analyzing a sequence of images, acquired at different test object planes at different z-positions. (Bravo-Zanoguera, M. E. Dissertation, University of California (2001)). An analog autofocus circuit generates voltages with respect to the images taken by the CCD camera at the different z-positions. During the z-movement the highest voltage of the circuit corresponds to the best focused image acquired at best focal position. After finding the best focal position, the image is acquired and the stage is moved to the next field, where the best focal position of the previous field is used as the origin of the new field focus search. (Bravo-Zanoguera, M. E. Dissertation, University of California (2001)). This method needs at least seven to nine different test object planes in order to obtain sufficient data and an accurate best focal position for cells and tissues on slides, coverslips and microtiter plates. (Bravo-Zanoguera; M., Massenbach, B., Kellner, A. & Price, J. High-performance autofocus circuit for biological microscopy. Review of Scientific Instruments 69, 3966-3977 (1998); and, Price, J. H. & Gough, D. A. Comparison of phase-contrast and fluorescence digital autofocus for scanning microscopy. Cytometry 16, 283-297 (1994)) In a sequence for incremental scanning, a focal function curve is obtained by the movement through the fields ( . . . , n−1, n, n+1 . . . ). The motion of the z-position is in control of a host computer.
The speed of the z-position movement is limited by vibrations originating from the acceleration of the stage during its start and stop motion. (Bravo-Zanoguera, M. E. Dissertation, University of California (2001)). Resultantly, sequential acquisition of the test object planes care done approximately at 3 Hz (3 fields/s), which results in a scanning time of 60 to 90 minutes for the entire slide.
Continuous scanning eliminates microscope stage acceleration and thus the scanning speed is improved while high resolution is maintained. (Bravo-Zanoguera, M. E. Dissertation, University of California (2001); Bravo-Zanoguera, M. E., Laris, C. A., Nguyen, L. K., Oliva, M. & Price, J. H. Dynamic autofocus for continuous-scanning time-delay-and-integration image acquisition in automated microscopy. Journal of Biomedical Optics 12, 34011/34011-34016 (2007); Bravo-Zanoguera, M. E. & Price, J. H. Simultaneous Multiplanar Image Acquisition in Light Microscopy. SPIE Proc. Optical Diagnostics of Biological Fluids and Advanced Techniques Analytical Cytology 3260 (1998); and Nguyen, L. K., Bravo-Zanoguera, M. E., Kellner, A. L. & Price, J. H. Magnification Corrected Optical Image Splitting Module for Simultaneous Multiplanar Acquisition. Proc. Of SPIE, Optical Diagnostics of Living Cells III 3921, 31-40 (2000)). To be able to scan a specimen by moving the stage at a constant velocity in the x- or y-directions, multiple test object planes are acquired simultaneously in the z-direction, a parallel autofocus algorithm determines the best focal position at each field and the focus is updated in a closed-loop-feedback correction manner several times per microscope field of view. (Bravo-Zanoguera, M. E. & Price, J. H. Simultaneous Multiplanar Image Acquisition in Light Microscopy. SPIE Proc. Optical Diagnostics of Biological Fluids and Advanced Techniques Analytical Cytology 3260 (1998)). To avoid the effects of start and stop acceleration, 1-D CCD sensors, also called line cameras, were first used in continuous scanning. (Castleman, K. R. The PSI automatic metaphase finder. J Radiat Res (Tokyo) 33 Suppl, 124-128 (1992); Shippey, G., Bayley, R., Farrow, S., Lutz, R. & Rutovitz, D. A fast interval processor (FIP) for cervical prescreening. Anal Quant Cytol 3, 9-16 (1981); Tucker, J. H., et al. Automated densitometry of cell populations in a continuous-motion imaging cell scanner. Applied Optics 26, 3315 (1987); and Tucker, J. H. & Shippey, G. Basic performance tests on the CERVIFIP linear array prescreener. Anal Quant Cytol 5, 129-137 (1983)). In this application the image readout from a line camera is synchronized with the stage motion to collect 2-D images. Especially for low-brightness specimens (e.g., fluorescently stained tissues), the time-delay-and-integrate (TDI) method of synchronizing the electronic representation of the image on an area CCD camera with optical image moving over the face of the CCD chip increases sensitivity. (Bravo-Zanoguera, M. E., Laris, C. A., Nguyen, L. K., Oliva, M. & Price, J. H. Dynamic autofocus for continuous-scanning time-delay-and-integration image acquisition in automated microscopy. Journal of Biomedical Optics 12, 34011/34011-34016 (2007); and Netten, H., Van Vliet, L. J., Boddeke, F. R., De Jong, P. & Young, I. T. A fast scanner for fluorescence microscopy using a 2-D CCD and time delayed integration. (1994)). The combination of continuous stage motion with multiplanar image acquisition considerably increases the scanning speed and allows for future continuous-scanning three-dimensional (3-D) imaging. Thus, multiplanar image acquisition replaces the serial process of sequential scanning by performing parallel calculations of focus and synchronous acquisition of all images. Multiplanar image acquisition can create a scanning speed which is just limited by the pixel acquisition rate of the camera, thereby enabling 2-D scanning cytometry that may reach the speed of 1-D flow cytometry. (Bravo-Zanoguera, M. E., Laris, C. A., Nguyen, L. K., Oliva, M. & Price, J. H. Dynamic autofocus for continuous-scanning time-delay-and-integration image acquisition in automated microscopy. Journal of Biomedical Optics 12, 34011/34011-34016 (2007)).
A multiplanar image acquisition system, also called the “volume camera”, is able to acquire multiple image planes of the z-direction simultaneously. (Bravo-Zanoguera, M. E., Laris, C. A., Nguyen, L. K., Oliva, M. & Price, J. H. Dynamic autofocus for continuous-scanning time-delay-and-integration image acquisition in automated microscopy. Journal of Biomedical Optics 12, 34011/34011-34016 (2007)). See also U.S. Pat. No. 6,640,014. In this regard, multiplanar image acquisition is based on the use of multiple CCD cameras, focused on multiple object planes. Such a stack of 2-D images represents one volume image of the specimen. Additional computational work combining the 2-D images provides 3-D imaging. The existence of such a volume camera with multiple object planes enables the rapid calculation of the best focal positions of a specimen during scanning.
However, the axial displacement of the object planes in a volume camera produces some variance in magnification, which alters the sampling rate of the imaging system and thus the performance of the autofocus algorithm. Such changes in magnification can be corrected by a computer, magnification correction optics, or if they are too small to alter the autofocus performance, can be neglected altogether. The lens, representing the microscope objective of the volume camera, magnifies the specimen, and multiple object planes within the specimen space are in focus simultaneously. Sensors are placed at different axial positions within the image space, where each sensor acquires one object plane and the corresponding information to calculate the in-focus position.
One such volume camera utilizes an array of nine optical fiber bundles coupled to an array of Time Delay and Integration (TDI) CCD cameras which are connected to an array of analog autofocus circuit boards. (Bravo-Zanoguera, M. E., Laris, C. A., Nguyen, L. K., Oliva, M. & Price, J. H. Dynamic autofocus for continuous-scanning time-delay-and-integration image acquisition in automated microscopy. Journal of Biomedical Optics 12, 34011/34011-34016 (2007)). The array of fiber bundles is arranged in a staircase pattern that provides the necessary axial displacement to acquire multiplanar images. However, due to the side-by-side arrangement of the fibers, there is spatial displacement which induces a spatial delay that has to be corrected by the computer. Furthermore, the volume camera is expensive to build and maintain because of the complex setup and supporting structure of the system as well as the optical inefficiency of the fiber bundles (60 percent transmittance). (Bravo-Zanoguera, M. E. & Price, J. H. Simultaneous Multiplanar Image Acquisition in Light Microscopy. SPIE Proc. Optical Diagnostics of Biological Fluids and Advanced Techniques Analytical Cytology 3260 (1998)).
These disadvantages led to a second design for simultaneous image acquisition in an image splitting system using a combination of beam splitters and mirrors to produce an eight channel multiplanar imaging system. See U.S. Pat. No. 6,839,469. The beam splitters are more effective and less fragile than the fiber bundle of the volume camera. In contrast to the fiber bundle system of the volume camera, spatial differences are prevented since the image originates from only one field of view. The system produces eight identical, non-inverted images at the optical output. Because of different z-positions, each object plane has a different magnification, which can be corrected by coupling each optical output channel through a zoom relay lens system, which focuses the image on one of the eight TDI CCD cameras. (Nguyen, L. K., Bravo-Zanoguera, M. E., Kellner, A. L. & Price, J. H. Magnification Corrected Optical Image Splitting Module for Simultaneous Multiplanar Acquisition. Proc. Of SPIE, Optical Diagnostics of Living Cells III 3921, 31-40 (2000)). In contrast to the fiber optical system of the volume camera, in which the fibers are glued to the sensors, the magnification optics can easily be added to the image splitting system. (Nguyen, L. K., Bravo-Zanoguera, M. E., Kellner, A. L. & Price, J. H. Magnification Corrected Optical Image Splitting Module for Simultaneous Multiplanar Acquisition. Proc. Of SPIE, Optical Diagnostics of Living Cells III 3921, 31-40 (2000)). Finally it is important to note that the image splitting system divides the light into eight channels, and also has glass-air interfaces. Thus, the intensity of each optical output to each individual camera is reduced significantly.
Multiplanar image acquisition speeds autofocus by enabling simultaneous testing of the relative sharpness of multiple axial planes. CCD cameras axially displaced relative to each other focus on different planes in the specimen space. This enables acquisition of multiple test object planes and thus allows for tracking focus and future 3-D imaging. An improved version of a volume camera, consisting of a series of mirrors and beam splitters, divides the volume image within the specimen into eight 2-D images. However, this system still suffers from optical aberrations due to huge amounts of glass. Correcting those aberrations is a very expensive and painstaking process.
The following specification proposes a solution to the sub-optimal performance of volume imaging devices in acquiring mutiplanar images and autofocusing, which utilizes the chromatic aberrations inherent in all optics and thereby reduces the amount beamsplitting optical glass and the number of relay lenses required to acquire a given number of focus axially-displaced test images.